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ABSTRACT 



Testing for critical thinking can be a challenge for new and experienced social 
work educators because critical thinking has no operational definition. Educational 
objectives commonly include the word “understanding,” but what does understanding 
mean and how do social work educators test for it? Bloom’s Taxonomy of Educational 
Objectives can help social work educators to think more precisely about critical thinking 
Bloom s taxonomy includes six knowledge levels: knowledge, comprehension, 
application, analysis, synthesis, and evaluation. The taxonomy levels are presented 
along with sample test items to show social work educators what kinds of test items tap 
higher knowledge levels. General guidelines for writing test questions are also 
presented. 



INTRODUCTION 
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Social work educators who wish to go beyond testing for memorization or “rote" 
learning will find that “critical thinking” has no clear operational definition and is too 
broad a term for practical use in the social work classroom. Critical thinking is a 
theoretical construct and we must measure behaviors that point to it instead of 
measuring it directly (Bloom, 1956; Wiggins & McTighe, 1998). This may be why 
educators can experience difficulty writing exam questions that do more than test for 
simple memorization. However, what is “understanding” and how do we recognize it in 
the social work classroom? A tool from the wider context of education, Bloom’s 
Taxonomy of Educational Objectives (Bloom, 1956), can help social work educators to 
think more precisely about what we test on our exams. Today we will examine the six 
knowledge levels of Bloom’s taxonomy of educational objectives, sample test items, and 
general test construction guidelines. 

Bloom’s Taxonomy of Educational Objectives 

The Taxonomy of Educational Objectives (1956), also called simply Bloom’s 
taxonomy, was written to add clarity and precision to the potentially challenging task of 
writing educational objectives. Educational objectives referring to students “knowing 
about” or “understanding” a topic are common in education, but are too broad to guide 
teaching and testing. Bloom’s taxonomy does not explicitly define critical thinking. 
Rather, it includes six knowledge levels that constitute critical thinking. The six 
knowledge levels can help us to clarify what critical thinking and “understanding” mean. 
The taxonomy applies to all materials in the social work curriculum, including the 
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knowledge courses (e.g., introductory courses), skill courses (e.g., practice methods) 
and field instruction. The taxonomy also can be utilized most testing formats including: 
completion, short answer, essays, written projects, multiple-choice, true/false, and 
matching formats. Although most of my testing examples utilize the multiple-choice 
format, they can easily be converted into the true/false, short answer, or essay formats. 

Figure one shows the six taxonomy knowledge levels arranged from the lowest 
level (knowledge) to the highest (evaluation), along with verbs associated with common 
student expectations (Bloom, 1971; Green, 1970, 1975). The knowledge levels are 
discrete and mastery of one level does not insure that students can automatically 
perform to higher levels. For example, simply knowing the elements of an intervention 
plan (knowledge) does not guarantee that students can produce one (synthesis) or critique 
one (evaluation). Teaching and testing are therefore linked, and teaching must support 
whether testing, for example, requires recognition of the name "Mary Richmond" 
(knowledge) or requires a critique of her contribution to social work (evaluation). 



Figure 1 . Bloom’s Taxonomv and Common Student Exoectations 




Knowledge Level 


Common Student Expectations 


Knowledge 

Comprehension 


Define, identify, state, list, differentiate, discriminate, recognize 
Explain, translate, interpret, match, extrapolate 


Application 

Analysis 

Synthesis 

Evaluation 


Construct, choose, predict, demonstrate 
Distinguish, separate, organize, infer, classify 
Compose, formulate, create, produce 
Debate, judge, critique, assess, compare 


Note: Adapted from: 


Bloom, 1971; Green, 1970, 1975. 
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Knowledge 

Knowledge is the lowest or simplest taxonomy level. Knowledge can be 
considered the facts or information students must learn. Knowledge is probably the 
most common level tested in higher education because instructors can simply use a 
textbook to determine what knowledge” students must learn and create exams targeted 
to a textbooks contents (Bloom, 1956). Creating knowledge test items can be as 
simple as removing the key word from a sentence and making it a choice among 
multiple choices, omitting a key word or phrase students must supply, or having 
students decide if a complete sentence is either true or false. 

Figure one includes eight sample knowledge items. Items 1-4, and item 7, 
require students to recognize material learned in class from among incorrect choices. 

In item five students must recognize the definition of a statistical test, and in item six 
students must recognize a research term translated into a dialogue. In item eight 
involves recognizing an example of a term leaned in class. Students can easily 
memorize material and correctly answer several of these items without “understanding” 
them (Gentile, 1990). After all, answering item one correctly does not mean students 

understand the purpose of NASW. Understanding refers to the next levels of Bloom’s 
taxonomy. 



Figure 2. Knowledge Level Test Items 
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1) What is the Social Work professional organization called? 

NASW ** CSW CSWE 

DSW AMA 

2) How much is the current food budget from the poverty line*? 

$100 $3.00 $.89 

$1.98 $1.19** 

3) Respectively, which choice shows the strongest positive, and strongest inverse 
correlations? 

-1.0, -.20 +.96, -.89** -.99, +1.0 

+.85, -.20 +.96, +.05 

4) What do progressives suggest the multiplication factor should be for the povertv line? 

2-9 .98 3.0 

5.6 4.0** 

5) Assuming ordinal data, which statistical test will determine if a sample and 
population have similar demographics? 

ANOVA Correlation coefficient Chi square *** 

Dependent f-test Frequency distribution 



6) Student: “The client was slapped 3 times in one day.” Supervisor: “How badly 
were they hurt?” The supervisor wants to know about: 

Frequency Intervals Duration 

Magnitude *** Precipitating events 

7) The majority of the social welfare budget is spent on which population group? 

Poor young adults Children The elderly *** 

The sick and disabled The disabled poor 

8) Given the context of material covered in class, which one is an example of 
Comparable Worth? 

Everyone pays the same proportion of income tax 
Jobs would be created so everyone would have work 
Different people are paid similarly for similar work** 

Women would have access to universal day care 
Wages are raised for everyone 
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Comprehension 
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Comprehension involves having students demonstrate they, for lack of a better 
word, understand material by doing more than simply memorizing and repeating it. 
Aspects of comprehension involve translation, interpretation, and extrapolation. 
Translation involves translation of learned material into a different, but correct, form. 
Interpretation involves summarization of learned material or examining material from 
different views. Extrapolation involves identifying trends and consequences. 

Figure three includes six comprehension test items. In item one and two, 
students must recall the conservative, liberal, progressive ideologies (Macht & Quam, 
1986), recognize the listed poverty theories (Schiller, 1998; Waxman, 1983) and match 
the theories with their political orientation. In items 3-6, students must recognize 
research and statistical terms translated into non-technical language. To correctly 
answer these items, students must do more than memorize and recall the association 
between statistical significance” and p<05. In item seven students must recognize the 
purpose of an element of an intervention or action plan. Creating comprehension 
questions are more difficult than creating knowledge questions because words or 

phrases cannot simply be removed from a sentence and hidden among multiple- 
choices. 
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Figure 3. Comprehension Level Test items 
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Which of these poverty theories is politically the most progressive? 

Genetic Inferiority Theory Functionalist Explanation Culture of Poverty 
Subclass Theory ** Orthodox Economic Theory 



2) The theories of poverty that are politically and economically the most progressive of 
the ones listed here are: 

Orthodox Economic Theory and Functionalist Explanation 
Culture of Poverty and Dual Labor Market Theory 
Subclass Explanation and Culture of Poverty 
Dual Labor Market Theory and Subclass Explanation *** 

Functionalist Explanation and Subclass Explanation 

3) “Statistical significance” means what? 

Decreased odds that results are due to “chance” ** 

The measures used have external validity 

The study has internal validity 

Increased odds that correct statistics were employed 

The study was conducted in a reliable manner 



4) The phrase “statistical significance” means: 

That a quasi -experimental study has internal validity 
Increased odds that “correct” statistics were employed 
The measures used to collect data have external validity 
Increased odds that results are not due to “chance” *** 
The study was conducted in a “reliable” manner 



5) Another way to describe the median score is calling it the: 

Midpoint *** Shape of the score distribution All of the above 

Average score Outlying scores in the score distribution 

6) What problem could happen if we interview people whose lives were saved 
by the police, but use police officers to conduct the interviews? 

Reductionism Operationalizing variables Reactivity *** 

Scientific determinism Logical consistency 

7) What should the plan of action include to help clients see they are making 
progress? 

Short & long term goals ** Partializing & prioritizing Objectivity & feedback 
Assessment & Listening & empathy 

information gathering 
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Application 

Application involves carry-over or transfer of learning to new situations or 
situations new to the students. Application is an important knowledge level for social 
work since our students must eventually apply what they learn to the problems clients 
will present. Creating different scenarios for teaching and testing involves advance 
preparation and can be time consuming. However, using the same example or scenario 
for teaching and testing means students may memorize the example without 
understanding it. 

In figure four, item one requires students to recall the definition of a causation (or 
feedback) loop and apply it to a scenario (e.g., cartoon characters) different from what I 
taught in class. Item two is similar and students must apply specific communication 
errors to a novel set of communications I did not utilize in class (Eriksen, 1979). 

Item three requires students to apply research concepts to an imaginary research study 
(Ellenbogen, 1993). In items 4-6, students must apply research concepts to new 
examples of those concepts and terms. Item seven is a sample essay question that 
requires students to apply their knowledge of President Reagan and his view of social 
welfare, to the New Deal. 
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Figure 4. Application Level Test Itpms 
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^ ?!? n . ald Duck doesr|,t think Goof y wiN be a good Disneyland tour guide and chooses 
Mickey instead. What is this called? 

Good Judgment Causation Loop ** Institutional Racism 

Institutional Discrimination All of tho abovo 



2) Respectively, what type of communication occurs in these three statements'? 

(a) 'Your child has acute exogenous bromadrosis but... 

(b) "Don't feel bad because... 

(c) "We have seminars for people who want to be better parents." 

Cliche, labeling, professionalitis Cliche, professionalitis labeling 
Labeling, professionalitis, cliche Professionalitis, labeling, cliches 
Professionalitis, cliches, labeling ** 

3) Identify the variables in the order they occur in this research study: “The effects of 
hanging on demonic possession in Quakers & Pilgrims" 

Dependent, Control, Independent Independent, Control, Dependent 
Independent, dependent, control ** Control, dependent, independent 
Control, independent, dependent 



4) Which choice shows the general relationship between cardiovascular fitness 
and blood pressure? 

- 1 -0*** +.20 +io 

-•25 +1.5 

5) If we change a p level from .05 to .001, the probability our results are NOT 
due to chance: 

Increases Decreases 

Increases, but external validity decreases Stavs the same 

Decreases, but reliability increases me same 

6) Which research question is probably explanatory? 

Which children ENJOY visiting Disneyland the most? 

What characterizes children who visit Disneyland? 

What are the top attractions in Disneyland? 

Why do children enjoy their visit to Disneyland? *** 

How many children visit Disneyland? 

7) Essay Question: 

Describe how the Social Security Act would have been different if 
President Reagan wrote it. 
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Analysis 

Analysis requires examination of parts or elements of concepts, analyzing the 
relationship between conclusions and evidence, organizing knowledge based on a 
principle, or making inferences based on data. In figure five, item one requires students 
to determine if a valid conclusion has been drawn from a table of statistical results 
(included on the exam). Item two has students infer what an American president 
thought based on legislation he proposed. Item three is similar and involves a popular 
writer on social policy issues. These types of items can be created utilizing the myriad of 
historical figures and events related to social work. In item four, students must utilize 
political ideologies to reorganize historical events related to poverty to determine which 
choice does not belong with the others. In item five students must utilize a contingency 
table to answer a question related to the safety of the drug thalidomide. I teach students 
about the decision to use thalidomide (and other more current drugs) but do not have 
students apply this example to the contingency table until the exam. If this exact 
material is taught in class in this form, students simply memorize the answers without 
understanding the concepts. 
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Figure 5. Analysis Level Test Items 
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1 ) 



Turn to the study on page 132 in your text. “All the perpetrators of sexual abuse 
in this study were first time offenders”. Does the data in table C support this 



Yes, the data supports the claim 

No, the data does not support the claim ** 

The data partially supports the claim 



2) Based on what President Roosevelt did in the New Deal, he probably thought: 
Social programs were giving too much money in aid to the poor 

There was a general lack of work ethic being caused by social welfare programs 
Helping the poor would make the economy worse than it already was at the time 
Poverty was linked to the economy so government should help the poor ** 
Poverty was solved by a strong individual work ethic 

3) If Charles Murray were allowed to reform welfare, TANF would probably: 

Have larger monthly cash benefits to help poor families raise their children 
Be eliminated*** 

Be connected more closely to private sector job creation 

Be more comprehensive in its aid to the poor 

Be forced to make benefits meet the national need standard 



4) Examined from an ideological perspective, which one does not belong with the others 
regarding the view of the poor? 

Elizabethan Poor Law - 1 601 Grant in Aid Programs 

War on Poverty Puritan Work Ethic ** 

Speenhamland Act 



REALITY 

Null is true Null is false 

Reject null 

DECISION 

Accept null 



5) Knowing what we do today, which block of the contingency table above contains 
the decision to give the drug thalidomide to pregnant mothers in the 1950s? 

A 

B 

C 

D** 

None of the above 



A. 


C. 


B. 


D. 
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Synthesis 
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Synthesis may be thought of as creativity because it involves the production of 
things that are new or unique. In social work education that can involve production of 
professional documents, intervention plans, communications, and the display of skills in 
unique combinations. The creative demonstration of learning and skills lends itself 
more to the essay format. Another way to assess if students can perform a skill or 
complete a task is with performance assessment where students actually perform a skill 
or complete a task (e.g., conduct a research study or client interview). 

Item one in figure six requires students to write an essay about proposing a new 
or improved way to handle American poverty. Students also identify weaknesses in 
their suggestions and view their work from a specific ideological position. Item two 
requires students to perform a client interview, a process recording and a psychosocial 
assessment to demonstrate communication and writing skills. Item three requires 
students to write and critique a research proposal. Any project or written assignment 
involves synthesis if it requires students to demonstrate social work skills. If we want to 

determine if our students can perform a skill, the best test may be to have them perform 
that skill. 
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Figure 6. Synthesis Level Test Itpms 



1) Multi-part Essay Question: Social Policy 

• Create a new way or improve how we currently deal with poverty in America 
Your suggestions must include a RATIONALE. 

• If your suggestions have been tried before in history (and they probably have in 
some form) how will you overcome the problems they encountered? 

• What are the problems with your suggestions and what could you do to overcome 
these problems? 

• Suggest an alternative policy that would be more progressive than what you just 
proposed and identify reasons why someone would support it. 



2) Written Project: Social Work Practice Methods 



• Produce an audiotape and process recording of a client interview you conduct with a 
classmate who portrays a client from your field placement. 






Write a psychosocial assessment of the client you interviewed, 
you did not have time for in the interview. 



Supplement material 



• Create an intervention plan based on the psychosocial assessment you did from 
your client interview. 



3) Written Project: Research Methods 

• Write a research proposal that includes all six steps of the research process. 

• Provide a justification for the steps in your proposal. For example, the cadence of 
your writing should be: 

“We chose a qualitative design because...” 

“We did choose sampling technique of X, BECAUSE it allowed us to....” 

• Grading of the research proposal has three criterion: (1 ) completeness, (2) if the 
study makes sense, (3) how well you justified and explained what you did. 
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Evaluation 



Evaluation requires students to make judgments based on external criteria or 
internal evidence. We must give students the external criteria and demonstrate how to 
utilize it to render judgments. Evaluative criteria exists forjudging the soundness of 
research projects and hypothesis, judging the correctness of communication and 
responding skills, judging the correctness of ethical decisions, etc. The social work 
code of ethics also lends itself to the evaluation knowledge level. Evaluation with 
internal evidence involves students using personal perspectives or value systems to 
evaluate situations. Students often demonstrate evaluation with written projects or 
class discussion since judgements from internal evidence are neither “right nor wrong.” 
In figure seven, item one requires students to judge which choice meets the 
criteria for the best (most behaviorally specific) operational definition. As with all the 
levels, students must recall all relevant definitions and evaluative criteria to answer the 
question. Item two is similar but students recognize “why” a definition is not 
operationalized. Item three has students evaluate the weakness with the sampling 
method employed in a research study. Item four requires students to use criteria for 
conducting ethical studies to evaluate why a research proposal was rejected (Williams, 
Unrau & Grinnell, 1998). Item five requires students to determine what will occur to the 
poverty line if the measure of poverty is changed. Item six has students determine what 
research design a study employs, while item seven requires students to make a 
judgement about when to share confidential information. 



Figure 7, Evaluation Level Test Items 
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1) Which of the following is the best operational definition? 

Caring and loving behavior Hitting behavior 

Homosexual behavior Slapping behavior ** 

Violent behavior 

2) Why is “showing love” a nominal definition? 

Behaviors are clearly specified No linguistic errors are present 

Thoughts and actions are clearly specified Specific behaviors are not clarified** 

There is no definition for love 

3) Turn to the study on page 184 in your textbook. The main SAMPLING problem in this 
study was that the sample: 

Self selected ** Did not match the population 

Was too large Was too small and too homogeneous 

Was too diverse 



4) What criteria did the agency most probably use to reject this research study? 

A social work agency rejects a proposed study because clients in crisis are 
assigned to a control group and will receive no service. 

Researchable Researcher interest area 

Relevant Ethical ** 

Feasible 

5) If we adopted the relative poverty measure in the USA, what would happen to the 
poverty count and line? 

Count-decrease; line-same Line-same; count-same 
Count-increase; line-increase ** Line-increase; count-decrease 
Line-same; count-increase 

6) Turn to the study on page 127 in your textbook. What is the design of this study? 

A) OXO** B) X 00 C) XI O D) O X O E) 00 X 00 

O O 00 X2 O 



7) You may share information without a clients written permission EXCEPT when the 
client: 

Has been physically abusing his child Has been verbally abusing his child ** 

Is HIV+, and not practicing safe sex None of the above 

Has threatened suicide and has a lethal plan 
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By now, some of you may be thinking that testing for critical thinking involves 
advance preparation. Well, it does. It is almost impossible to achieve anything near to 
critical thinking if we write our exams the night before needed while trying to recall what 
was taught. There are too many factors that go into writing good test questions for them 
to be written in haste. Figure eight includes some helpful guidelines to consider when 
writing your test questions. Even if we write our test questions with care, test questions 
do not always perform up to standards or do what we intended them to do. My students 
are usually willing to share what they thought were problems with disputed questions, 
even if the problem was that they simply answered the question incorrectly (sometimes 
their only criteria for evaluating a test question). Figure nine includes some major 
considerations for testing our test questions. 

Creating tests using Bloom’s taxonomy takes time and intention but the effort is 
well worth it. The taxonomy has helped me clarify my intentions in teaching and testing 
and in creating course materials that match my learning expectations for students. I can 
also make my tests more challenging by teaching and testing to higher knowledge 
levels instead of by burying essential information within other information not intended 
for testing, or by making finer discriminations between response choices. Students often 
say my course material is “tough but fair” (high praise from students). I encourage you 
to try utilizing Bloom’s taxonomy to create your teaching and testing materials. 



o 
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Figure 8. General Principles for Writing Test Questions 

• Avoid long questions, they take more time to read and increase test time 

• Shorten questions and answers as much as possible 

• Do not make longest answer correct! 

• Do not make ALL OF THE ABOVE answers always correct 

• Rotate correct answer choices so students do not react to patterns 

• Avoid questions in the negative, or mark clearly with EXCEPT 

• True false questions should be either true or false, and become too easy when the 
false statements are too clearly false. 

• May format questions to have a definition in the question and names as the 
answers, or the reverse. 

• Have examples ready for application questions that are close to the concept but are 

not the same as those utilized in class. 

Figure 9. General Principles for Testing Test Questions 



• Evaluating norm referenced and criterion referenced test questions is different. Norm 
referenced testing compares student performance to other students (normal curve). 
Criterion referenced measurement compares student performance to a standard. 

• The Item Difficulty Index shows how many students answered questions correctly. 
Too much error (over 70%) may indicate a problem with an item, too little error (less 
than 30%) may be acceptable depending on the goal of the testing. 

• Ask students to explain how they answered the questions will show the instructor if a 
question tapped the intended knowledge level. 

• Different errors require different types of correction. The errors may not be apparent 
until the student explains how they tried to answer the question. 

• Content validity is critical for criterion referenced testing and is established by 

determining if content tested links to content covered. 
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